Incorporating Function Values into Quasi-Newton Updates

The traditional quasi-Newton method for updating the approximate Hessian is based on the change in the gradient of the objective function. This paper describes a new update method that incorporates also the change in the value of the function. The method effectively uses a cubic approximation of the objective function to better approximate its directional second derivative. The cubic approximation is adjusted when necessary to ensure the positive definiteness of the approximating Hessian matrix. For the BFGS method this results in an average improvement of about 5% in performance. This improvement is modest, but can be obtained with a correspondingly modest change to the BFGS Hessian update algorithm. The best improvement is obtained with a line search algorithm that uses modified Wolfe conditions; however, in practice, the traditional line search algorithms can be used almost as effectively.